Skip to content

Test passing fixtures and minimum supported VS Code - #145

Merged
rcosta358 merged 8 commits into
codex/issue-131-vscode-smoke-testsfrom
codex/issue-132-passing-min-vscode
Oct 7, 2026
Merged

rcosta358 merged 8 commits into
codex/issue-131-vscode-smoke-testsfrom
codex/issue-132-passing-min-vscode

Conversation

@rcosta358

@rcosta358 rcosta358 commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Add an isolated passing workspace and assert the first diagnostic result after Verify. Run both passing and failing fixtures on stable and the minimum supported VS Code (1.82.0) for every pull request, main, and reusable release check.

Validated fixture failure reporting, workflow coverage, lint, types, and extension installation.

Depends on #144. Closes #132.

🤖 Generated with Codex

@CatarinaGamboa CatarinaGamboa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three things, mostly about what happens when something goes wrong.

Reviewed with Claude Code (reviewer + adversarial agents per PR, findings checked against the code before posting).

Comment thread client/src/test/smoke.test.ts Outdated
const nextFixtureDiagnostics = () => new Promise<LJDiagnostic[]>((resolve) => {
const subscription = api.onDiagnostics((diagnostics) => {
if (diagnostics.some(d => d.type === 'refinement-error' && path.resolve(d.file) === uri.fsPath)) {
const matches = passing ? diagnostics.length === 0

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A regression shows up as a bare 120s timeout. This wait only resolves when the result is the expected one (empty for passing, a refinement error for failing). If the passing fixture starts getting an error, or the failing one gets none, the promise never resolves. The asserts below (including assert.deepEqual(diagnostics, []) on line 46, which can't fail) are never reached, and the failure is a timeout with no diagnostics in the output.

Suggest resolving on the first diagnostics notification for this run and asserting on it, so a failure prints what actually came back.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e5b2580: waits capture the first diagnostics notification and assert its contents; crashes fail immediately with fixture/status details. Both fixtures passed on stable and minimum VS Code, and a focused harness verified unexpected results fail promptly.

Comment thread .github/workflows/test.yml Outdated
strategy:
fail-fast: false
matrix:
version: ${{ (github.event_name == 'pull_request' || github.ref == 'refs/heads/main') && fromJSON('["stable", "minimum"]') || fromJSON('["stable"]') }}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two things with this job:

  1. This PR removes the fork-only if that Add real VS Code integration smoke test #144 had on integration. So for same-repo PRs every push runs it twice: the push run (stable), plus the PR run (stable + minimum).
  2. Release tags only test stable. publish.yml calls this workflow with event push and ref refs/tags/v*, so neither branch of this condition matches. Releases then never test the minimum supported VS Code version.

Suggest bringing the if back, but still letting minimum run for PRs and tags. For example, put the decision in the matrix and the if, so that pushes to branches run stable, while pushes to main, tags (startsWith(github.ref, 'refs/tags/')) and fork PRs run both.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e5b2580: integration skips duplicate same-repository PR runs; main, release tags, and fork PRs test stable plus minimum. Required Checks remains unconditional.

@CatarinaGamboa CatarinaGamboa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The minimum (1.82.0) leg never runs for same-repo PRs: the matrix only adds it for pull_request/main/tags, but the job if: skips same-repo pull_request events, and branch pushes only get stable. E.g. latest runs on #145/#146/#147 only show "VS Code integration (stable)". Maybe include minimum on push events too?

codex and others added 4 commits October 6, 2026 23:17
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
@rcosta358

Copy link
Copy Markdown
Collaborator Author

Addressed the missing minimum-version coverage: integration now uses a static [stable, minimum] matrix and runs on every PR, main, and reusable release check. Both passing and failing fixtures are tested, and the earlier first-notification/error-reporting fixes remain in place.

Independent review, workflow checks, extension installation, and both integration legs in the final CI run passed.

@rcosta358
rcosta358 added this pull request to stack #148 October 7, 2026 11:17
@rcosta358
rcosta358 merged commit fcd3f41 into main Oct 7, 2026
3 of 6 checks passed
rcosta358 added a commit that referenced this pull request Oct 7, 2026
Adds real VS Code coverage for webview readiness, diagnostics/context
messages, and Stop, Start, and Restart. Checks process termination and
verification after Restart, and fixes a shutdown race that could clear
the newly started server.

Validated both fixtures locally and in CI on stable and VS Code 1.82.0;
lint, types, and installation passed.

Depends on #145. Closes #133.

Generated by Codex.

---------

Co-authored-by: Codex <noreply@openai.com>
rcosta358 added a commit that referenced this pull request Oct 7, 2026
## Description
Closes #134.

Add weekly and manual Windows/macOS runs for client and server unit
tests plus VS Code stable integration tests, with logs uploaded on
failure. The diff contains only `platform-tests.yml`.

## Related Issues
Depends on #146. The existing prerequisite PRs now form one chain
through #145, #144, #143, #141, #142, #140, #139, #138, and #137 to
main. The schedule becomes active when merged to the default branch.

Validation: 32 client tests, 24 server tests, lint, production/test
TypeScript checks, and extension installation passed. Stable and minimum
VS Code integration passed in [PR
CI](https://github.com/liquid-java/vscode-liquidjava/actions/runs/37541530280).
Fresh [Windows/macOS
validation](https://github.com/liquid-java/vscode-liquidjava/actions/runs/37541530341)
passed on the final commit, including both unit-test suites and VS Code
integration.

🤖 Generated with [Codex](https://openai.com/codex/)

---------

Co-authored-by: Codex <noreply@openai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Testing related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a passing fixture and test the oldest supported VS Code

3 participants